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DETAILED ACTION 
Appeal Brief 

1 . In response to the Office Action mailed January 3, 2007, Applicants filed an 
Appeal Brief filed on May 31, 2007. After reviewing the remarks, Applicant's arguments 
were persuasive. Prosecution will be reopened based on new grounds of rejections and 
the Office regrets any inconvenience. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-6, 8, 11, 14-15, 17-19, 21-26, 28, 31, 34-35, 37-39 and 41-47 are 

rejected under 35 U.S.C. 103(a) as being unpatentable over Adams et al. (PGPUB 
2004/0008209), hereinafter referenced as Adams in view of Belrose (PGPUB 
2003/0144843). 

Regarding claims 1 and 21, Adams discloses a system and method for 
cataloguing electronic information (cataloguing; column 3, paragraphs 0079), 
comprising: 

an electronic device that captures audio/video data corresponding to a 
photographic target, said audio/video data (audio and video) including a narration 
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provided by a narrator to identify respective subject matter locations in said audio/video 
data (narrative; column 1, paragraph 0002); 

generating labels that correspond to said respective subject matter locations in 
said audio/video data (column 6, paragraph 0111); and 

a label manager (labeling) that manages a label mode for generating and storing 
said labels, said label manager also controlling a label search mode for utilizing said 
labels to locate said respective subject matter locations in said audio/video data 
(location; column 6, paragraph 011 1), but does not specifically teach a speech 
recognition engine. 

Belrose disclose a system and method for cataloguing electronic information 
(catalogue data; column 9, paragraph 0180) comprising: 

an electronic device that captures audio/video data corresponding to a 
photographic target, said audio/video data (audio and video) including a narration 
provided by a narrator to identify respective subject matter locations in said audio/video 
data (narrator; column 6, paragraphs 0106-0107); and 

a speech recognition engine (speech interface system) that automatically 
performs a speech recognition process upon said narration to generate labels (label) 
that correspond to said respective subject matter locations in said audio/video data 
(feature; column 6, paragraphs 0106-0107 and column 3, paragraphs 0045-0047 with 
column 2, paragraph 0020), for collecting user-interest information regarding a picture. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify to modify Adams method and system wherein it 
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provides a speech recognition engine, as taught by Belrose, to provide associated 
information and to facilitate the provision of user-relevant information for such pictures 
(column 2, paragraph 0021). 

Regarding claims 2 and 22, Adams discloses a system and method wherein 
said electronic device is implemented as an audio/video camcorder device (video 
recorder; column 9, paragraph 0140). 

Regarding claims 3 and 23, Adams in view of Belrose teach everything as 
recited in claim 1. In addition Belrose teach a system and method wherein said speech 
recognition engine is configured in a simplified configuration that efficiently compares 
said narration with acoustic models to identify phone strings that represent said 
narration, said speech recognition engine (speech interface system) referencing a 
compact dictionary to look up recognized vocabulary words that correspond to said 
phone strings, said speech recognition engine utilizing a limited set of recognition 
grammar to form said recognized vocabulary words into said labels (label) that are 
supported by said speech recognition engine (column 6, paragraphs 0106-0107) 

Regarding claims 4 and 24, Adams teaches a method and system wherein said 
label manager initially instructs said electronic device to enter a real-time label mode for 
creating and storing said labels, said electronic device concurrently capturing said 
audio/video data and said narration after said label manager instructs said electronic 
device to enter said real-time label mode (column 1, paragraph 0008-0009). 

Regarding claims 5 and 25, Adams in view of Belrose teach everything as 
recited in claim 1. In addition Belrose teach a system and method wherein said 
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electronic device enters a real-time label mode in response to a verbal label- mode 
command from a system user, said verbal label-mode command being recognized and 
provided to said label manager by said speech recognition engine (column 6, 
paragraphs 0106-0107). 

Regarding claims 6 and 26, Adams discloses a system and method wherein 
said speech recognition engine automatically generates said labels as said electronic 
device captures said audio/video data and said narration (automatically; column 3, 
paragraph 0080). 

Regarding claims 8 and 28, Adams discloses a system and method wherein 
said label manager stores said labels during a real-time label mode, said labels being 
stored along with meta-information that associates each of said respective subject 
matter locations to a corresponding one of said labels (meta-data; column 3, paragraph 
0079). 

Regarding claims 11 and 31, Adams discloses a system and method wherein 
said speech recognition engine automatically generates said labels by analyzing said 
audio/video data and said narration as said electronic device plays back said 
audio/video data and said narration (column 3, paragraph 0079). 

Regarding claims 14 and 34, Adams in view of Belrose teach everything as 
recited in claim 1. In addition Belrose teach a system and method wherein said label 
manager coordinates a label validation procedure for validating said labels in response 
to verbal validation commands from a system user, said verbal validation commands 
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being recognized and provided to said label manager by said speech recognition engine 
(voice dialog; column 3, paragraphs 0045-0047). 

Regarding claims 15 and 35, Adams discloses a system and method wherein 
said label manager stores said labels in a non-real-time label mode, said labels being 
stored along with meta-information that associates each of said respective subject 
matter locations to a corresponding one of said labels (meta-data; column 3, paragraph 
0079). 

Regarding claims 17 and 37, Adams discloses a system and method wherein 
said label manager generates a label-search GUI on a display of said electronic device, 
a system user viewing said labels and corresponding representative images from said 
audio/video data for selecting a search label (display; column 4, paragraph 0093. In 
addition, Belrose teaches a GUI display (GUI browser; column 1, paragraph 0002 with 
column 6, paragraph 0106) 

Regarding claims 18 and 38, Adams in view of Belrose teach everything as 
recited in claim 1 . In addition Belrose teach a system and method wherein a system 
user selects a search label by issuing a verbal search-label command, said verbal 
search-label command being recognized and provided to said label manager by said 
speech recognition engine (column 3, paragraphs 0045-0047 with column 6, 
paragraphs 0106-0107). 

Regarding claims 19 and 39, Adams discloses a system and method wherein 
said label manager instructs said electronic device to automatically locate and retrieve a 
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specific one of said respective subject matter locations in response to a system user 
selecting a search label (automatically; column 3, paragraph 0080). 

Claim 41 is directed toward a computer readable medium to implement or 
execute the system of claim 1 , and is similar in scope and content of claim 1 , therefore, 
claim 41 is rejected under similar rationale. 

Claims 42-47 are directed toward a system to implement or execute the method 
of claim 21 , and is similar in scope and content of claim 21 , therefore, claims 42-47 is 
rejected under similar rationale. 

4. Claims 7, 12, 27 and 32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Adams in view of Belrose and in further view of Nicholson et al. 
(PGPUB 2002/0067859), hereinafter referenced as Nicholason. 

Regarding claims 7 and 27, Adams discloses a system and method for 
cataloguing electronic information, but does not specifically teach wherein a post 
processor performs a post-processing procedure upon said labels in a real- time label 
mode, said post-processing procedure including a validation procedure using one or 
more confidence measures to eliminate invalid labels that fail to satisfy pre-determined 
validation criteria. 

Nicholason teaches a system and method wherein a post processor performs a 
post-processing procedure upon said labels in a real- time label mode, said post- 
processing procedure including a validation procedure (validate) using one or more 
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confidence measures (confidence level) to eliminate invalid labels that fail to satisfy pre- 
determined validation criteria (column 2, paragraph 0012), to examine for possible 
errors. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Adams in view of Belrose system and method as 
described above, to allow the user to preview the results when the data structure 
confidence threshold is changed, which allows the user to optimize the number of words 
that need examination for possible error correction (column 10, paragraph 0101). 

Regarding claims 12 and 32, Adams discloses a system and method for 
cataloguing electronic information, but does not specifically teach wherein a post 
processor performs a post-processing procedure upon said labels in a non- real-time 
label mode, said post-processing procedure including a validation procedure using one 
or more confidence measures to eliminate invalid labels that fail to satisfy pre- 
determined validation criteria. 

Nicholason teaches a system and method wherein a post processor performs a 
post-processing procedure upon said labels in a non- real-time label mode, said post- 
processing procedure including a validation procedure using one or more confidence 
measures to eliminate invalid labels that fail to satisfy pre-determined validation criteria 
(column 2, paragraph 0012), to examine for possible errors. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Adams in view of Belrose system and method as 
described above, to allow the user to preview the results when the data structure 
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confidence threshold is changed, which allows the user to optimize the number of words 
that need examination for possible error correction (column 10, paragraph 0101). 

5. Claims 9-10, 13, 16, 20, 29-30, 33, 36 and 40 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Adams in view of Belrose and in further view of 
Frohlich et al. (PBPUB 2004/0037540), hereinafter referenced as Frohllich. 

Regarding claims 9 and 29, Adams discloses a system and method for 
cataloguing electronic information, but does not specifically teach wherein said 
electronic device initially captures said audio/video data and said narration prior to 
entering said label mode. 

Frohlich teaches a system and method comprising a digital camcorder (column 1 , 
paragraph 0007 and column 6, paragraph 0146) wherein the sound can be captured 
immediately before, at the same time or immediately after capturing the image (column 
1, paragraph 0008 with columns 4-5, paragraphs 0118-0120 and column 6, pargraph 
0136). Frohlich teaches narrative data and catalogue data (column 5, paragraph 0120 
and column 6, paragraph 0137), to allow a plurality of photograph images to be 
associated with audio. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Adams in view of Belrose system and method as 
described above, to have an electronic photograph album that allows multiple audio 
tracks to be associated with a single image (column 2, paragraphs 0016-0026). Also to 
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provide for a hierarchical architecture of a plurality of audio photographs having different 
levels, whereby audio photographs can be arranged or grouped into sets of audio 
photographs, such that within a set, a plurality of audio photographs can be collected 
according to similar theme and the set can have its own layers and segments. 

Regarding claims 10 and 30, it is interpreted and rejected for the same reasons 
as set for in claims 9 and 29. In addition Frohlich discloses a system and method 
wherein said label manager instructs said electronic device to enter a non-real-time 
label mode for creating and storing said labels, said electronic device responsively 
retrieving and playing back said audio/video data and said narration (activating play; 
column 7, paragraph 0156). 

Regarding claims 13 and 33, it is interpreted and rejected for the same reasons 
as set for in claims 9 and 29. In addition Frohlich discloses a system and method 
wherein said label manager coordinates a label validation procedure for validating said 
labels, said label manager generating a validation graphical user interface upon a 
display of said electronic device for a system user to interactively evaluate (edit), delete 
(remove), and edit said labels (column 7, paragraph 0155 and column 7, paragraph 
0161). 

Regarding claims 16 and 36, it is interpreted and rejected for the same reasons 
as set for in claims 9 and 29. In addition Frohlich discloses a system and method 
wherein said label manager instructs said electronic device to enter said label search 
mode during which a system user interactively selects a search label for performing a 
label search procedure to locate a specific one of said respective subject matter 
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locations corresponding to said search label (activate play; column 7, paragraphs 0156 
and 0160). 

Regarding claims 20 and 40, it is interpreted and rejected for the same reasons 
as set for in claims 9 and 29. In addition Frohlich discloses a system and method 
wherein said electronic device automatically plays back a specific retrieved one of said 
respective subject matter locations from said audio/video data for viewing by said 
system user (playback; column 7, paragraph 0156) 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Aschbrenner et al. (USPN 6,463,205) disclose a personalized video story 
production apparatus and method. 

• Davis et al. (USPN 7,101 ,144) disclose associating data with images in imaging 
systems. 

• Vetterli et al. (PGPUB 2002/0075282) disclose automated annotation of a view. 

• Balabanovic et al. (PGPUB 2005/0283741) disclose a method and apparatus for 
storytelling with digital photographs. 
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7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jakieda R. Jackson whose telephone number is 571- 
272-7619. The examiner can normally be reached on Monday-Friday from 5:30am- 
2:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached on 571-272-7843. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




JRJ 

September 3, 2007 



DAVID HUDSPETH 
SUPERVISORY PATENT EXAMINER 
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